Move similarity analysis in chess programs

In June 2011, the International Computer Games Association (ICGA) disqualified Vasik Rajlich and his Rybka chess program for plagiarism and breaking their rules on originality in their events from 2006-10. One primary basis for this came from a painstaking code comparison, using the source code of Fruit and the object code of Rybka, which found the selection of evaluation features in the programs to be almost the same, much more than expected by chance. In his brief defense, Rajlich indicated his opinion that move similarity testing was a superior method of detecting misappropriated entries. Later commentary by both Rajlich and his defenders reiterated the same, and indeed the ICGA Rules themselves specify move similarity as an example reason for why the tournament director would have warrant to request a source code examination. We report on data obtained from move-similarity testing. The principal dataset here consists of over 8000 positions and nearly 100 independent engines. We comment on such issues as: the robustness of the methods (upon modifying the experimental conditions), whether strong engines tend to play more similarly than weak ones, and the observed Fruit/Rybka move-similarity data. 1. History and background on derivative programs in computer chess Computer chess has seen a number of derivative programs over the years. One of the first was the incident in the 1989 World Microcomputer Chess Championship (WMCCC), in which Quickstep was disqualified due to the program being “a copy of the program Mephisto Almeria” in all important areas. See ICCA Journal 12/4. The conclusion here was rather clear, and left little room for contention; later cases would not always be so easy. The emergence of open-source software being run on general-purpose hardware, in particular Crafty in the mid 1990s, led to another prickly situation, with the 1996 WMCCC seeing Gunda participate, though it was derivative of Crafty, who was also in the field. It should be noted that the Gunda authors fully acknowledged the debt to Crafty, and there were side issues with Gunda being from the host institution. See the editorial in ICCA Journal 20/1. The 2003 World Computer Chess Championship (WCCC) saw the competitor List disqualified due to a problem with timely inspection of program code (following a request by the tournament director). The author was later exonerated from any specific wrongdoing, upon code inspection (at a later date) by an Preprint submitted to Elsevier November 1, 2013 outside examiner. See [14] and also ICCA Journal 28/1. A successor to List, namely Loop in the 2007 WCCC, is currently under investigation by the ICGA as a possible Fruit derivative. Although never disqualified, the ElChinito author Eugenio Castillo Jimenez admitted re-using Crafty source code in an open letter dated Aug 2004 [16]. The WCCC entrant named Chinito in 2002 and 2003 was never directly linked, and Jimenez participated in later ICGA events, with XieXie in Chinese Chess. The 2006 WCCC saw another disqualification, with the LION++ team being dismissed for not duly informing the tournament committee that their program was derivative of the open-source Fruit (which had competed in 2005, and would permissibly provide the basis for cluster-based entrants in 2007 and 2008). They had acknowledged Fruit in their program notes, but not as required on the entry form, and not received the necessary permission from Fruit’s author Fabien Letouzey in any case. Two inspectors concluded that LION++ was a “close derivative” of Fruit, and thus it was disqualified. See ICCA Journal 29/2. Another example is Squarknll being refused admission to the 2010 WCCC on the grounds that it was a barely-modified version of the open-source program RobboLito (itself of rather dubious pedigree). See [18] for more. In other competitive games, such as computer go, one can give the example of Silver Igo, alternatively named KCC Igo (or KCC Paduk), and commercialised as Ginsei Igo, which was claimed to derive from Handtalk [29], written by Chen Zhixing. This was reported on in 2000 by Keene [17]. Zhixing has given various evidence for this [27], and also concerning a program named Hamlet, alternatively called Hit, AI-Igo 7, or Medusa (see [28]). Due to unresolved issues from a decade previous, KCC Paduk was denied entry to the 2008 International Computer Games Championship [5]. 1.1. Other computer chess derivatives When expanded to programs that do not compete in live tournaments, the list of derivatives is probably too long to enumerate, though Ron Murawski has a partial list at [19]. We should mention here in particular two examples involving Rybka, both of which have some bearing on the events recounted below. The first was the release of Strelka, initially in mid-2007, and then later in a source code version in early 2008. While initially being rather permissive toward Strelka, the Rybka author Vasik Rajilch eventually declared Strelka 2.0 to be a virtual clone of Rybka 1.0, and this seems to be adequately substantiated via outside examination and general consensus. Secondly, in 2009 an open-source program called IPPOLIT (later morphed into RobboLito and IvanHoe) was released, and Rajlich indicated that it was a reverse-engineered clone of Rybka 3. Subsequent analysis by Watkins (see [24]) couched this view more cautiously, saying that undoubtedly the makers of IPPOLIT had reverse-engineered Rybka 3 as a starting point, but that vari1Ciancarini and Favini mention Rybka/Strelka in [CF], but with no definitive conclusion.